<?xml version='1.0' encoding="utf-8"?>
<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html lang="en" xml:lang="en" xmlns="http://www.w3.org/1999/xhtml">
  <head>
    <link href="cpip.css" rel="stylesheet" type="text/css" />
    <title>File: /Users/paulross/dev/linux/linux-3.13/arch/x86/include/asm/rwsem.h</title>
  </head>
  <body>
    <h1>File: /Users/paulross/dev/linux/linux-3.13/arch/x86/include/asm/rwsem.h</h1>
    <p>Green shading in the line number column
means the source is part of the translation unit, red means it is conditionally excluded.
Highlighted line numbers link to the translation unit page. Highlighted macros link to
the macro page.</p>
    <pre><a name="1" /><span class="True">       1:</span> <span class="k">/* rwsem.h: R/W semaphores implemented using XADD/CMPXCHG for i486+</span>
<a name="2" /><span class="True">       2:</span> <span class="k"> *</span>
<a name="3" /><span class="True">       3:</span> <span class="k"> * Written by David Howells (dhowells@redhat.com).</span>
<a name="4" /><span class="True">       4:</span> <span class="k"> *</span>
<a name="5" /><span class="True">       5:</span> <span class="k"> * Derived from asm-x86/semaphore.h</span>
<a name="6" /><span class="True">       6:</span> <span class="k"> *</span>
<a name="7" /><span class="True">       7:</span> <span class="k"> *</span>
<a name="8" /><span class="True">       8:</span> <span class="k"> * The MSW of the count is the negated number of active writers and waiting</span>
<a name="9" /><span class="True">       9:</span> <span class="k"> * lockers, and the LSW is the total number of active locks</span>
<a name="10" /><span class="True">      10:</span> <span class="k"> *</span>
<a name="11" /><span class="True">      11:</span> <span class="k"> * The lock count is initialized to 0 (no active and no waiting lockers).</span>
<a name="12" /><span class="True">      12:</span> <span class="k"> *</span>
<a name="13" /><span class="True">      13:</span> <span class="k"> * When a writer subtracts WRITE_BIAS, it&apos;ll get 0xffff0001 for the case of an</span>
<a name="14" /><span class="True">      14:</span> <span class="k"> * uncontended lock. This can be determined because XADD returns the old value.</span>
<a name="15" /><span class="True">      15:</span> <span class="k"> * Readers increment by 1 and see a positive value when uncontended, negative</span>
<a name="16" /><span class="True">      16:</span> <span class="k"> * if there are writers (and maybe) readers waiting (in which case it goes to</span>
<a name="17" /><span class="True">      17:</span> <span class="k"> * sleep).</span>
<a name="18" /><span class="True">      18:</span> <span class="k"> *</span>
<a name="19" /><span class="True">      19:</span> <span class="k"> * The value of WAITING_BIAS supports up to 32766 waiting processes. This can</span>
<a name="20" /><span class="True">      20:</span> <span class="k"> * be extended to 65534 by manually checking the whole MSW rather than relying</span>
<a name="21" /><span class="True">      21:</span> <span class="k"> * on the S flag.</span>
<a name="22" /><span class="True">      22:</span> <span class="k"> *</span>
<a name="23" /><span class="True">      23:</span> <span class="k"> * The value of ACTIVE_BIAS supports up to 65535 active processes.</span>
<a name="24" /><span class="True">      24:</span> <span class="k"> *</span>
<a name="25" /><span class="True">      25:</span> <span class="k"> * This should be totally fair - if anything is waiting, a process that wants a</span>
<a name="26" /><span class="True">      26:</span> <span class="k"> * lock will go to the back of the queue. When the currently active lock is</span>
<a name="27" /><span class="True">      27:</span> <span class="k"> * released, if there&apos;s a writer at the front of the queue, then that and only</span>
<a name="28" /><span class="True">      28:</span> <span class="k"> * that will be woken up; if there&apos;s a bunch of consequtive readers at the</span>
<a name="29" /><span class="True">      29:</span> <span class="k"> * front, then they&apos;ll all be woken up, but no other readers will be.</span>
<a name="30" /><span class="True">      30:</span> <span class="k"> */</span>
<a name="31" /><span class="True">      31:</span> 
<a name="32" /><span class="True">      32:</span> <span class="f">#</span><span class="n">ifndef</span> <a href="cpu.c_macros_noref.html#_X0FTTV9YODZfUldTRU1fSF8w"><span class="b">_ASM_X86_RWSEM_H</span></a>
<a name="33" /><span class="True">      33:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_noref.html#_X0FTTV9YODZfUldTRU1fSF8w"><span class="b">_ASM_X86_RWSEM_H</span></a>
<a name="34" /><span class="True">      34:</span> 
<a name="35" /><span class="False">      35:</span> <span class="f">#</span><span class="n">ifndef</span> <a href="cpu.c_macros_ref.html#_X0xJTlVYX1JXU0VNX0hfMA__"><span class="b">_LINUX_RWSEM_H</span></a>
<a name="36" /><span class="False">      36:</span> <span class="f">#</span><span class="n">error</span> <span class="e">&quot;please don&apos;t include asm/rwsem.h directly, use linux/rwsem.h instead&quot;</span>
<a name="37" /><span class="True">      37:</span> <span class="f">#</span><span class="n">endif</span>
<a name="38" /><span class="True">      38:</span> 
<a name="39" /><span class="True">      39:</span> <span class="f">#</span><span class="n">ifdef</span> <a href="cpu.c_macros_ref.html#_X19LRVJORUxfX18w"><span class="b">__KERNEL__</span></a>
<a name="40" /><span class="True">      40:</span> <span class="f">#</span><span class="n">include</span> <span class="f">&lt;</span><span class="m">asm</span><span class="f">/</span><span class="m">asm</span><span class="f">.</span><span class="b">h</span><span class="f">&gt;</span>
<a name="41" /><span class="True">      41:</span> 
<a name="42" /><span class="True">      42:</span> <span class="k">/*</span>
<a name="43" /><span class="True">      43:</span> <span class="k"> * The bias values and the counter type limits the number of</span>
<a name="44" /><span class="True">      44:</span> <span class="k"> * potential readers/writers to 32767 for 32 bits and 2147483647</span>
<a name="45" /><span class="True">      45:</span> <span class="k"> * for 64 bits.</span>
<a name="46" /><span class="True">      46:</span> <span class="k"> */</span>
<a name="47" /><span class="True">      47:</span> 
<a name="48" /><span class="True">      48:</span> <span class="f">#</span><span class="n">ifdef</span> <a href="cpu.c_macros_ref.html#_Q09ORklHX1g4Nl82NF8w"><span class="b">CONFIG_X86_64</span></a>
<a name="49" /><span class="True">      49:</span> <span class="f">#</span> <span class="n">define</span> <a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX01BU0tfMA__"><span class="b">RWSEM_ACTIVE_MASK</span></a>        <span class="c">0xffffffffL</span>
<a name="50" /><span class="False">      50:</span> <span class="f">#</span><span class="n">else</span>
<a name="51" /><span class="False">      51:</span> <span class="f">#</span> <span class="n">define</span> <a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX01BU0tfMA__"><span class="b">RWSEM_ACTIVE_MASK</span></a>        <span class="c">0x0000ffffL</span>
<a name="52" /><span class="True">      52:</span> <span class="f">#</span><span class="n">endif</span>
<a name="53" /><span class="True">      53:</span> 
<a name="54" /><span class="True">      54:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_noref.html#_UldTRU1fVU5MT0NLRURfVkFMVUVfMA__"><span class="b">RWSEM_UNLOCKED_VALUE</span></a>        <span class="c">0x00000000L</span>
<a name="55" /><span class="True">      55:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX0JJQVNfMA__"><span class="b">RWSEM_ACTIVE_BIAS</span></a>        <span class="c">0x00000001L</span>
<a name="56" /><span class="True">      56:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_ref.html#_UldTRU1fV0FJVElOR19CSUFTXzA_"><span class="b">RWSEM_WAITING_BIAS</span></a>        <span class="f">(</span><span class="f">-</span><a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX01BU0tfMA__"><span class="b">RWSEM_ACTIVE_MASK</span></a><span class="f">-</span><span class="c">1</span><span class="f">)</span>
<a name="57" /><span class="True">      57:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX1JFQURfQklBU18w"><span class="b">RWSEM_ACTIVE_READ_BIAS</span></a>        <a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX0JJQVNfMA__"><span class="b">RWSEM_ACTIVE_BIAS</span></a>
<a name="58" /><span class="True">      58:</span> <span class="f">#</span><span class="n">define</span> <a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX1dSSVRFX0JJQVNfMA__"><span class="b">RWSEM_ACTIVE_WRITE_BIAS</span></a>        <span class="f">(</span><a href="cpu.c_macros_ref.html#_UldTRU1fV0FJVElOR19CSUFTXzA_"><span class="b">RWSEM_WAITING_BIAS</span></a> <span class="f">+</span> <a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX0JJQVNfMA__"><span class="b">RWSEM_ACTIVE_BIAS</span></a><span class="f">)</span>
<a name="59" /><span class="True">      59:</span> 
<a name="60" /><span class="True">      60:</span> <span class="k">/*</span>
<a name="61" /><span class="True">      61:</span> <span class="k"> * lock for reading</span>
<a name="62" /><span class="True">      62:</span> <span class="k"> */</span>
<a name="63" /><span class="True">      63:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">__down_read</span><span class="f">(</span><span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="64" /><span class="True">      64:</span> <span class="f">{</span>
<a name="65" /><span class="True">      65:</span>     <span class="m">asm</span> <span class="m">volatile</span><span class="f">(</span><span class="e">&quot;# beginning down_read\n\t&quot;</span>
<a name="66" /><span class="True">      66:</span>              <a href="cpu.c_macros_ref.html#_TE9DS19QUkVGSVhfMA__"><span class="b">LOCK_PREFIX</span></a> <a href="cpu.c_macros_ref.html#_X0FTTV9JTkNfMA__"><span class="b">_ASM_INC</span></a> <span class="e">&quot;(%1)\n\t&quot;</span>
<a name="67" /><span class="True">      67:</span>              <span class="k">/* adds 0x00000001 */</span>
<a name="68" /><span class="True">      68:</span>              <span class="e">&quot;  jns        1f\n&quot;</span>
<a name="69" /><span class="True">      69:</span>              <span class="e">&quot;  call call_rwsem_down_read_failed\n&quot;</span>
<a name="70" /><span class="True">      70:</span>              <span class="e">&quot;1:\n\t&quot;</span>
<a name="71" /><span class="True">      71:</span>              <span class="e">&quot;# ending down_read\n\t&quot;</span>
<a name="72" /><span class="True">      72:</span>              <span class="f">:</span> <span class="e">&quot;+m&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span>
<a name="73" /><span class="True">      73:</span>              <span class="f">:</span> <span class="e">&quot;a&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">)</span>
<a name="74" /><span class="True">      74:</span>              <span class="f">:</span> <span class="e">&quot;memory&quot;</span><span class="f">,</span> <span class="e">&quot;cc&quot;</span><span class="f">)</span><span class="f">;</span>
<a name="75" /><span class="True">      75:</span> <span class="f">}</span>
<a name="76" /><span class="True">      76:</span> 
<a name="77" /><span class="True">      77:</span> <span class="k">/*</span>
<a name="78" /><span class="True">      78:</span> <span class="k"> * trylock for reading -- returns 1 if successful, 0 if contention</span>
<a name="79" /><span class="True">      79:</span> <span class="k"> */</span>
<a name="80" /><span class="True">      80:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">int</span> <span class="b">__down_read_trylock</span><span class="f">(</span><span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="81" /><span class="True">      81:</span> <span class="f">{</span>
<a name="82" /><span class="True">      82:</span>     <span class="m">long</span> <span class="b">result</span><span class="f">,</span> <span class="b">tmp</span><span class="f">;</span>
<a name="83" /><span class="True">      83:</span>     <span class="m">asm</span> <span class="m">volatile</span><span class="f">(</span><span class="e">&quot;# beginning __down_read_trylock\n\t&quot;</span>
<a name="84" /><span class="True">      84:</span>              <span class="e">&quot;  mov          %0,%1\n\t&quot;</span>
<a name="85" /><span class="True">      85:</span>              <span class="e">&quot;1:\n\t&quot;</span>
<a name="86" /><span class="True">      86:</span>              <span class="e">&quot;  mov          %1,%2\n\t&quot;</span>
<a name="87" /><span class="True">      87:</span>              <span class="e">&quot;  add          %3,%2\n\t&quot;</span>
<a name="88" /><span class="True">      88:</span>              <span class="e">&quot;  jle	     2f\n\t&quot;</span>
<a name="89" /><span class="True">      89:</span>              <a href="cpu.c_macros_ref.html#_TE9DS19QUkVGSVhfMA__"><span class="b">LOCK_PREFIX</span></a> <span class="e">&quot;  cmpxchg  %2,%0\n\t&quot;</span>
<a name="90" /><span class="True">      90:</span>              <span class="e">&quot;  jnz	     1b\n\t&quot;</span>
<a name="91" /><span class="True">      91:</span>              <span class="e">&quot;2:\n\t&quot;</span>
<a name="92" /><span class="True">      92:</span>              <span class="e">&quot;# ending __down_read_trylock\n\t&quot;</span>
<a name="93" /><span class="True">      93:</span>              <span class="f">:</span> <span class="e">&quot;+m&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;=&amp;a&quot;</span> <span class="f">(</span><span class="b">result</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;=&amp;r&quot;</span> <span class="f">(</span><span class="b">tmp</span><span class="f">)</span>
<a name="94" /><span class="True">      94:</span>              <span class="f">:</span> <span class="e">&quot;i&quot;</span> <span class="f">(</span><a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX1JFQURfQklBU18w"><span class="b">RWSEM_ACTIVE_READ_BIAS</span></a><span class="f">)</span>
<a name="95" /><span class="True">      95:</span>              <span class="f">:</span> <span class="e">&quot;memory&quot;</span><span class="f">,</span> <span class="e">&quot;cc&quot;</span><span class="f">)</span><span class="f">;</span>
<a name="96" /><span class="True">      96:</span>     <span class="m">return</span> <span class="b">result</span> <span class="f">&gt;=</span> <span class="c">0</span> <span class="f">?</span> <span class="c">1</span> <span class="f">:</span> <span class="c">0</span><span class="f">;</span>
<a name="97" /><span class="True">      97:</span> <span class="f">}</span>
<a name="98" /><span class="True">      98:</span> 
<a name="99" /><span class="True">      99:</span> <span class="k">/*</span>
<a name="100" /><span class="True">     100:</span> <span class="k"> * lock for writing</span>
<a name="101" /><span class="True">     101:</span> <span class="k"> */</span>
<a name="102" /><span class="True">     102:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">__down_write_nested</span><span class="f">(</span><span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">,</span> <span class="m">int</span> <span class="b">subclass</span><span class="f">)</span>
<a name="103" /><span class="True">     103:</span> <span class="f">{</span>
<a name="104" /><span class="True">     104:</span>     <span class="m">long</span> <span class="b">tmp</span><span class="f">;</span>
<a name="105" /><span class="True">     105:</span>     <span class="m">asm</span> <span class="m">volatile</span><span class="f">(</span><span class="e">&quot;# beginning down_write\n\t&quot;</span>
<a name="106" /><span class="True">     106:</span>              <a href="cpu.c_macros_ref.html#_TE9DS19QUkVGSVhfMA__"><span class="b">LOCK_PREFIX</span></a> <span class="e">&quot;  xadd      %1,(%2)\n\t&quot;</span>
<a name="107" /><span class="True">     107:</span>              <span class="k">/* adds 0xffff0001, returns the old value */</span>
<a name="108" /><span class="True">     108:</span>              <span class="e">&quot;  test &quot;</span> <a href="cpu.c_macros_ref.html#_X19BU01fU0VMXzA_"><span class="b">__ASM_SEL</span></a><span class="f">(</span><span class="f">%</span><span class="b">w1</span><span class="f">,</span><span class="f">%</span><span class="b">k1</span><span class="f">)</span> <span class="e">&quot;,&quot;</span> <a href="cpu.c_macros_ref.html#_X19BU01fU0VMXzA_"><span class="b">__ASM_SEL</span></a><span class="f">(</span><span class="f">%</span><span class="b">w1</span><span class="f">,</span><span class="f">%</span><span class="b">k1</span><span class="f">)</span> <span class="e">&quot;\n\t&quot;</span>
<a name="109" /><span class="True">     109:</span>              <span class="k">/* was the active mask 0 before? */</span>
<a name="110" /><span class="True">     110:</span>              <span class="e">&quot;  jz        1f\n&quot;</span>
<a name="111" /><span class="True">     111:</span>              <span class="e">&quot;  call call_rwsem_down_write_failed\n&quot;</span>
<a name="112" /><span class="True">     112:</span>              <span class="e">&quot;1:\n&quot;</span>
<a name="113" /><span class="True">     113:</span>              <span class="e">&quot;# ending down_write&quot;</span>
<a name="114" /><span class="True">     114:</span>              <span class="f">:</span> <span class="e">&quot;+m&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;=d&quot;</span> <span class="f">(</span><span class="b">tmp</span><span class="f">)</span>
<a name="115" /><span class="True">     115:</span>              <span class="f">:</span> <span class="e">&quot;a&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;1&quot;</span> <span class="f">(</span><a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX1dSSVRFX0JJQVNfMA__"><span class="b">RWSEM_ACTIVE_WRITE_BIAS</span></a><span class="f">)</span>
<a name="116" /><span class="True">     116:</span>              <span class="f">:</span> <span class="e">&quot;memory&quot;</span><span class="f">,</span> <span class="e">&quot;cc&quot;</span><span class="f">)</span><span class="f">;</span>
<a name="117" /><span class="True">     117:</span> <span class="f">}</span>
<a name="118" /><span class="True">     118:</span> 
<a name="119" /><span class="True">     119:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">__down_write</span><span class="f">(</span><span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="120" /><span class="True">     120:</span> <span class="f">{</span>
<a name="121" /><span class="True">     121:</span>     <span class="b">__down_write_nested</span><span class="f">(</span><span class="b">sem</span><span class="f">,</span> <span class="c">0</span><span class="f">)</span><span class="f">;</span>
<a name="122" /><span class="True">     122:</span> <span class="f">}</span>
<a name="123" /><span class="True">     123:</span> 
<a name="124" /><span class="True">     124:</span> <span class="k">/*</span>
<a name="125" /><span class="True">     125:</span> <span class="k"> * trylock for writing -- returns 1 if successful, 0 if contention</span>
<a name="126" /><span class="True">     126:</span> <span class="k"> */</span>
<a name="127" /><span class="True">     127:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">int</span> <span class="b">__down_write_trylock</span><span class="f">(</span><span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="128" /><span class="True">     128:</span> <span class="f">{</span>
<a name="129" /><span class="True">     129:</span>     <span class="m">long</span> <span class="b">result</span><span class="f">,</span> <span class="b">tmp</span><span class="f">;</span>
<a name="130" /><span class="True">     130:</span>     <span class="m">asm</span> <span class="m">volatile</span><span class="f">(</span><span class="e">&quot;# beginning __down_write_trylock\n\t&quot;</span>
<a name="131" /><span class="True">     131:</span>              <span class="e">&quot;  mov          %0,%1\n\t&quot;</span>
<a name="132" /><span class="True">     132:</span>              <span class="e">&quot;1:\n\t&quot;</span>
<a name="133" /><span class="True">     133:</span>              <span class="e">&quot;  test &quot;</span> <a href="cpu.c_macros_ref.html#_X19BU01fU0VMXzA_"><span class="b">__ASM_SEL</span></a><span class="f">(</span><span class="f">%</span><span class="b">w1</span><span class="f">,</span><span class="f">%</span><span class="b">k1</span><span class="f">)</span> <span class="e">&quot;,&quot;</span> <a href="cpu.c_macros_ref.html#_X19BU01fU0VMXzA_"><span class="b">__ASM_SEL</span></a><span class="f">(</span><span class="f">%</span><span class="b">w1</span><span class="f">,</span><span class="f">%</span><span class="b">k1</span><span class="f">)</span> <span class="e">&quot;\n\t&quot;</span>
<a name="134" /><span class="True">     134:</span>              <span class="k">/* was the active mask 0 before? */</span>
<a name="135" /><span class="True">     135:</span>              <span class="e">&quot;  jnz          2f\n\t&quot;</span>
<a name="136" /><span class="True">     136:</span>              <span class="e">&quot;  mov          %1,%2\n\t&quot;</span>
<a name="137" /><span class="True">     137:</span>              <span class="e">&quot;  add          %3,%2\n\t&quot;</span>
<a name="138" /><span class="True">     138:</span>              <a href="cpu.c_macros_ref.html#_TE9DS19QUkVGSVhfMA__"><span class="b">LOCK_PREFIX</span></a> <span class="e">&quot;  cmpxchg  %2,%0\n\t&quot;</span>
<a name="139" /><span class="True">     139:</span>              <span class="e">&quot;  jnz	     1b\n\t&quot;</span>
<a name="140" /><span class="True">     140:</span>              <span class="e">&quot;2:\n\t&quot;</span>
<a name="141" /><span class="True">     141:</span>              <span class="e">&quot;  sete         %b1\n\t&quot;</span>
<a name="142" /><span class="True">     142:</span>              <span class="e">&quot;  movzbl       %b1, %k1\n\t&quot;</span>
<a name="143" /><span class="True">     143:</span>              <span class="e">&quot;# ending __down_write_trylock\n\t&quot;</span>
<a name="144" /><span class="True">     144:</span>              <span class="f">:</span> <span class="e">&quot;+m&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;=&amp;a&quot;</span> <span class="f">(</span><span class="b">result</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;=&amp;r&quot;</span> <span class="f">(</span><span class="b">tmp</span><span class="f">)</span>
<a name="145" /><span class="True">     145:</span>              <span class="f">:</span> <span class="e">&quot;er&quot;</span> <span class="f">(</span><a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX1dSSVRFX0JJQVNfMA__"><span class="b">RWSEM_ACTIVE_WRITE_BIAS</span></a><span class="f">)</span>
<a name="146" /><span class="True">     146:</span>              <span class="f">:</span> <span class="e">&quot;memory&quot;</span><span class="f">,</span> <span class="e">&quot;cc&quot;</span><span class="f">)</span><span class="f">;</span>
<a name="147" /><span class="True">     147:</span>     <span class="m">return</span> <span class="b">result</span><span class="f">;</span>
<a name="148" /><span class="True">     148:</span> <span class="f">}</span>
<a name="149" /><span class="True">     149:</span> 
<a name="150" /><span class="True">     150:</span> <span class="k">/*</span>
<a name="151" /><span class="True">     151:</span> <span class="k"> * unlock after reading</span>
<a name="152" /><span class="True">     152:</span> <span class="k"> */</span>
<a name="153" /><span class="True">     153:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">__up_read</span><span class="f">(</span><span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="154" /><span class="True">     154:</span> <span class="f">{</span>
<a name="155" /><span class="True">     155:</span>     <span class="m">long</span> <span class="b">tmp</span><span class="f">;</span>
<a name="156" /><span class="True">     156:</span>     <span class="m">asm</span> <span class="m">volatile</span><span class="f">(</span><span class="e">&quot;# beginning __up_read\n\t&quot;</span>
<a name="157" /><span class="True">     157:</span>              <a href="cpu.c_macros_ref.html#_TE9DS19QUkVGSVhfMA__"><span class="b">LOCK_PREFIX</span></a> <span class="e">&quot;  xadd      %1,(%2)\n\t&quot;</span>
<a name="158" /><span class="True">     158:</span>              <span class="k">/* subtracts 1, returns the old value */</span>
<a name="159" /><span class="True">     159:</span>              <span class="e">&quot;  jns        1f\n\t&quot;</span>
<a name="160" /><span class="True">     160:</span>              <span class="e">&quot;  call call_rwsem_wake\n&quot;</span> <span class="k">/* expects old value in %edx */</span>
<a name="161" /><span class="True">     161:</span>              <span class="e">&quot;1:\n&quot;</span>
<a name="162" /><span class="True">     162:</span>              <span class="e">&quot;# ending __up_read\n&quot;</span>
<a name="163" /><span class="True">     163:</span>              <span class="f">:</span> <span class="e">&quot;+m&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;=d&quot;</span> <span class="f">(</span><span class="b">tmp</span><span class="f">)</span>
<a name="164" /><span class="True">     164:</span>              <span class="f">:</span> <span class="e">&quot;a&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;1&quot;</span> <span class="f">(</span><span class="f">-</span><a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX1JFQURfQklBU18w"><span class="b">RWSEM_ACTIVE_READ_BIAS</span></a><span class="f">)</span>
<a name="165" /><span class="True">     165:</span>              <span class="f">:</span> <span class="e">&quot;memory&quot;</span><span class="f">,</span> <span class="e">&quot;cc&quot;</span><span class="f">)</span><span class="f">;</span>
<a name="166" /><span class="True">     166:</span> <span class="f">}</span>
<a name="167" /><span class="True">     167:</span> 
<a name="168" /><span class="True">     168:</span> <span class="k">/*</span>
<a name="169" /><span class="True">     169:</span> <span class="k"> * unlock after writing</span>
<a name="170" /><span class="True">     170:</span> <span class="k"> */</span>
<a name="171" /><span class="True">     171:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">__up_write</span><span class="f">(</span><span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="172" /><span class="True">     172:</span> <span class="f">{</span>
<a name="173" /><span class="True">     173:</span>     <span class="m">long</span> <span class="b">tmp</span><span class="f">;</span>
<a name="174" /><span class="True">     174:</span>     <span class="m">asm</span> <span class="m">volatile</span><span class="f">(</span><span class="e">&quot;# beginning __up_write\n\t&quot;</span>
<a name="175" /><span class="True">     175:</span>              <a href="cpu.c_macros_ref.html#_TE9DS19QUkVGSVhfMA__"><span class="b">LOCK_PREFIX</span></a> <span class="e">&quot;  xadd      %1,(%2)\n\t&quot;</span>
<a name="176" /><span class="True">     176:</span>              <span class="k">/* subtracts 0xffff0001, returns the old value */</span>
<a name="177" /><span class="True">     177:</span>              <span class="e">&quot;  jns        1f\n\t&quot;</span>
<a name="178" /><span class="True">     178:</span>              <span class="e">&quot;  call call_rwsem_wake\n&quot;</span> <span class="k">/* expects old value in %edx */</span>
<a name="179" /><span class="True">     179:</span>              <span class="e">&quot;1:\n\t&quot;</span>
<a name="180" /><span class="True">     180:</span>              <span class="e">&quot;# ending __up_write\n&quot;</span>
<a name="181" /><span class="True">     181:</span>              <span class="f">:</span> <span class="e">&quot;+m&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;=d&quot;</span> <span class="f">(</span><span class="b">tmp</span><span class="f">)</span>
<a name="182" /><span class="True">     182:</span>              <span class="f">:</span> <span class="e">&quot;a&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;1&quot;</span> <span class="f">(</span><span class="f">-</span><a href="cpu.c_macros_ref.html#_UldTRU1fQUNUSVZFX1dSSVRFX0JJQVNfMA__"><span class="b">RWSEM_ACTIVE_WRITE_BIAS</span></a><span class="f">)</span>
<a name="183" /><span class="True">     183:</span>              <span class="f">:</span> <span class="e">&quot;memory&quot;</span><span class="f">,</span> <span class="e">&quot;cc&quot;</span><span class="f">)</span><span class="f">;</span>
<a name="184" /><span class="True">     184:</span> <span class="f">}</span>
<a name="185" /><span class="True">     185:</span> 
<a name="186" /><span class="True">     186:</span> <span class="k">/*</span>
<a name="187" /><span class="True">     187:</span> <span class="k"> * downgrade write lock to read lock</span>
<a name="188" /><span class="True">     188:</span> <span class="k"> */</span>
<a name="189" /><span class="True">     189:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">__downgrade_write</span><span class="f">(</span><span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="190" /><span class="True">     190:</span> <span class="f">{</span>
<a name="191" /><span class="True">     191:</span>     <span class="m">asm</span> <span class="m">volatile</span><span class="f">(</span><span class="e">&quot;# beginning __downgrade_write\n\t&quot;</span>
<a name="192" /><span class="True">     192:</span>              <a href="cpu.c_macros_ref.html#_TE9DS19QUkVGSVhfMA__"><span class="b">LOCK_PREFIX</span></a> <a href="cpu.c_macros_ref.html#_X0FTTV9BRERfMA__"><span class="b">_ASM_ADD</span></a> <span class="e">&quot;%2,(%1)\n\t&quot;</span>
<a name="193" /><span class="True">     193:</span>              <span class="k">/*</span>
<a name="194" /><span class="True">     194:</span> <span class="k">              * transitions 0xZZZZ0001 -&gt; 0xYYYY0001 (i386)</span>
<a name="195" /><span class="True">     195:</span> <span class="k">              *     0xZZZZZZZZ00000001 -&gt; 0xYYYYYYYY00000001 (x86_64)</span>
<a name="196" /><span class="True">     196:</span> <span class="k">              */</span>
<a name="197" /><span class="True">     197:</span>              <span class="e">&quot;  jns       1f\n\t&quot;</span>
<a name="198" /><span class="True">     198:</span>              <span class="e">&quot;  call call_rwsem_downgrade_wake\n&quot;</span>
<a name="199" /><span class="True">     199:</span>              <span class="e">&quot;1:\n\t&quot;</span>
<a name="200" /><span class="True">     200:</span>              <span class="e">&quot;# ending __downgrade_write\n&quot;</span>
<a name="201" /><span class="True">     201:</span>              <span class="f">:</span> <span class="e">&quot;+m&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span>
<a name="202" /><span class="True">     202:</span>              <span class="f">:</span> <span class="e">&quot;a&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">)</span><span class="f">,</span> <span class="e">&quot;er&quot;</span> <span class="f">(</span><span class="f">-</span><a href="cpu.c_macros_ref.html#_UldTRU1fV0FJVElOR19CSUFTXzA_"><span class="b">RWSEM_WAITING_BIAS</span></a><span class="f">)</span>
<a name="203" /><span class="True">     203:</span>              <span class="f">:</span> <span class="e">&quot;memory&quot;</span><span class="f">,</span> <span class="e">&quot;cc&quot;</span><span class="f">)</span><span class="f">;</span>
<a name="204" /><span class="True">     204:</span> <span class="f">}</span>
<a name="205" /><span class="True">     205:</span> 
<a name="206" /><span class="True">     206:</span> <span class="k">/*</span>
<a name="207" /><span class="True">     207:</span> <span class="k"> * implement atomic add functionality</span>
<a name="208" /><span class="True">     208:</span> <span class="k"> */</span>
<a name="209" /><span class="True">     209:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">void</span> <span class="b">rwsem_atomic_add</span><span class="f">(</span><span class="m">long</span> <span class="b">delta</span><span class="f">,</span> <span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="210" /><span class="True">     210:</span> <span class="f">{</span>
<a name="211" /><span class="True">     211:</span>     <span class="m">asm</span> <span class="m">volatile</span><span class="f">(</span><a href="cpu.c_macros_ref.html#_TE9DS19QUkVGSVhfMA__"><span class="b">LOCK_PREFIX</span></a> <a href="cpu.c_macros_ref.html#_X0FTTV9BRERfMA__"><span class="b">_ASM_ADD</span></a> <span class="e">&quot;%1,%0&quot;</span>
<a name="212" /><span class="True">     212:</span>              <span class="f">:</span> <span class="e">&quot;+m&quot;</span> <span class="f">(</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">)</span>
<a name="213" /><span class="True">     213:</span>              <span class="f">:</span> <span class="e">&quot;er&quot;</span> <span class="f">(</span><span class="b">delta</span><span class="f">)</span><span class="f">)</span><span class="f">;</span>
<a name="214" /><span class="True">     214:</span> <span class="f">}</span>
<a name="215" /><span class="True">     215:</span> 
<a name="216" /><span class="True">     216:</span> <span class="k">/*</span>
<a name="217" /><span class="True">     217:</span> <span class="k"> * implement exchange and add functionality</span>
<a name="218" /><span class="True">     218:</span> <span class="k"> */</span>
<a name="219" /><span class="True">     219:</span> <span class="m">static</span> <span class="m">inline</span> <span class="m">long</span> <span class="b">rwsem_atomic_update</span><span class="f">(</span><span class="m">long</span> <span class="b">delta</span><span class="f">,</span> <span class="m">struct</span> <span class="b">rw_semaphore</span> <span class="f">*</span><span class="b">sem</span><span class="f">)</span>
<a name="220" /><span class="True">     220:</span> <span class="f">{</span>
<a name="221" /><span class="True">     221:</span>     <span class="m">return</span> <span class="b">delta</span> <span class="f">+</span> <a href="cpu.c_macros_ref.html#_eGFkZF8w"><span class="b">xadd</span></a><span class="f">(</span><span class="f">&amp;</span><span class="b">sem</span><span class="f">-&gt;</span><span class="b">count</span><span class="f">,</span> <span class="b">delta</span><span class="f">)</span><span class="f">;</span>
<a name="222" /><span class="True">     222:</span> <span class="f">}</span>
<a name="223" /><span class="True">     223:</span> 
<a name="224" /><span class="True">     224:</span> <span class="f">#</span><span class="n">endif</span> <span class="k">/* __KERNEL__ */</span>
<a name="225" /><span class="True">     225:</span> <span class="f">#</span><span class="n">endif</span> <span class="k">/* _ASM_X86_RWSEM_H */</span>
<a name="226" /><span class="True">     226:</span> </pre>
  </body>
</html>
